MARDAN , Pakistan -LRB- CNN -RRB- -- A family of 18 Pakistani men , women and children trudges down a dirt road toward a refugee camp .

These children are among the thousands of refugees this week at the Jalozai camp in western Pakistan .

Adolescent girls carry infants on their hips , while the men lug bundles of belongings on their backs .

`` Come , stay close to me , '' said one woman wrapped in brightly colored robes , speaking to three children trailing behind her .

`` This one is empty , '' a white-bearded Pakistani police officer tells the family , pointing toward a tent .

The women and children scramble under the canvas flap , as Salar Khan explains what led his family to flee to Mardan .

`` Mortars destroyed three houses in my village , '' he said . `` It was dangerous . A piece of shrapnel almost pierced my child 's leg . ''

Khan said his family left their home Wednesday morning in Sultanwas , a town in Buner district .

Now , they are living in Mardan 's rapidly growing tent city of more than 1,400 other displaced Pakistanis . Five days ago , it was an empty field . Watch as CNN 's Ivan Watson tours a refugee camp ''

Khan 's family has joined tens of thousands of other Pakistanis fleeing south to escape the escalating conflict between the military and Taliban militants in northwestern Pakistan . Meanwhile , columns of Pakistani troops in military trucks head in the opposite direction , hauling field guns north toward the conflict zone .

Pakistani families have fled the area any way they can : on foot , by hitching rides on the back of trucks and by stowing their belongings on the roofs of cars .

As fighting has spread from the districts of Buner and Lower Dir to the Taliban stronghold in the Swat Valley , camps for displaced people are cropping up across northwest Pakistan .

The United Nations said the new exodus is exacerbating an already existing humanitarian crisis . Since August , the U.N. has registered more than 500,000 Pakistanis forced to flee their homes by fighting in other northwestern parts of the country .

`` Last year ... 4 million people worldwide lost their homes , out of which you have half a million displaced in Pakistan , '' said Manuel Bessler , a top U.N. official in Islamabad .

Bessler spoke on a rooftop , overlooking the sprawling Jalozai refugee camp in western Pakistan .

Until recently , the camp housed refugees from neighboring Afghanistan . The Afghans are now gone , replaced by more then 49,000 Pakistanis .

Administrators are preparing space for 35,000 others . With help from U.N. agencies , the Pakistani government and other aid organizations , residents get access to medical care , children 's schools and training programs to teach them how to rebuild their damaged homes if and when they get to return .

Tensions have been building in the Jalozai camp . Two months ago , Pakistani police shot and killed one demonstrator after residents protested , blocking roads , throwing stones and demanding compensation for homes damaged by the fighting .

This week , a crowd of several hundred agitated men gathered at the entrance , angry about a delay of several days in the monthly distribution of food aid . Some accused camp administrators of corruption , allegations that aid workers have denied .

`` The wheat we 've been given is substandard , and people are getting sick instead of being fed , '' said one man named Gulzada .

`` Our houses have been destroyed , '' said another man called Anwar . `` There 's no tea , no sugar , no wheat , no lentils . All that we have are the clothes we are wearing . ''

A fresh wave of displaced Pakistanis will only aggravate tensions , said Bessler , the U.N. official .

`` This is a factor that is destabilizing not only in the camp but in the country as a whole , '' he warned .

Only a fraction of the hundreds of thousands of displaced Pakistanis are ending up in camps . Many more have settled with host families or have resorted to paying rent in other cities .

The influx of ethnic Pashtuns from northwest Pakistan upset the delicate demographic balance last month in the port city of Karachi . That led to ethnic clashes between Pashtuns and the resident Muhajir community , resulting in the deaths of more than 30 people .

Many more Pakistanis are unable to leave the conflict area , according to Sebastian Brack , a spokesman for the International Committee of the Red Cross in Islamabad .

`` There is a serious humanitarian crisis under way , '' Brack said . `` There is serious fighting going on . There will be massive displacement . Because of the curfew , -LSB- many -RSB- have not been able to leave yet . ''

In this moment of crisis , some homeless Pakistanis are turning to a higher power .

`` Whenever it is God 's will , we will go back to our homes , '' says Mohammed Munir , an elderly man who fled with his family from the Buner district to the new camp in Mardan three days ago . `` And we pray to Allah that he will protect us . It 's up to Allah . We ca n't do anything . ''

The man kneeled and prayed in the grass outside the entrance of a tent that his family now calls home .

@highlight

Tens of thousands flee south to escape conflict between military and Taliban

@highlight

Refugee camps for the displaced are cropping up across northwest Pakistan

@highlight

Latest exodus is exacerbating humanitarian crisis , U.N. says

@highlight

Tensions have been building in Jalozai camp in western Pakistan